1. Field of the Invention
The present invention relates to a method for correcting defective combined pixel signals generated from pixel signals of phase difference pixels used for focus detection and an image capture device.
2. Description Related to the Prior Art
An autofocusing (AF) function for performing automatic focus adjustment has become a standard feature of image capture devices such as digital cameras and surveillance cameras. Autofocusing functions include a contrast AF operation that performs focus adjustment to make the contrast of the image maximum and a phase detection AF operation that performs the focus adjustment based on a phase difference caused by parallax. The contrast AF operation requires finding an inflection point of the contrast while a taking lens is moved, whereas the phase detection AF operation enables fast autofocusing because it detects the focus state at each lens position.
The image capture device which adopted the phase detection AF operation uses an image sensor (solid-state imaging device) which has pixels (hereinafter referred to as the normal pixels) of symmetrical structure and pixels (hereinafter referred to as the phase difference pixels) of asymmetrical structure used for focus detection. The normal pixels are used for obtaining pixel signals (pixel values) for producing an image (normal image) taken in the normal mode. The phase difference pixels are used for obtaining information of phase difference used for the phase difference AF operation. The phase difference pixel has an opening at the position shifted to the left or right relative to that of the normal pixel, and selectively receives light incident on a right portion or a left portion of a photodiode.
In the case where the taking lens is set in the focusing position, the waveform of the pixel signal read out from the right phase difference pixel group for receiving the light from the right side coincides with the waveform of the pixel signal read out from the left phase difference pixel group for receiving the light from the left side, and there is no phase difference between them. In the case where the taking lens is out of the focusing position, a phase difference appears in the waveforms of the two pixel signals in accordance with defocus. The sequences of the phases of the two pixel signal waveforms which correspond to the front and rear focuses, respectively, differ from each other. Therefore the information of the phase difference is extracted to set the taking lens to the focusing position.
It is common for the image sensor having the phase difference pixels to produce the normal image with the use of the phase difference pixels in addition to the normal pixels because pixel missing occurs at the positions corresponding to the phase difference pixels in the normal image in the case where the normal image is produced based only on the pixel signals from the normal pixels. However, the pixel signals obtained, without correction, from the phase difference pixels cannot be used as the pixel signals for the normal image because the sensitivity and the like of the phase difference pixels differ from those of the normal pixels. In order to produce the normal image, the pixel having the pixel signal obtained from the phase difference pixel is treated as a defective pixel and the pixel signal is subjected to a correction process. For example, in the case of a color normal image, a method (hereinafter referred to as the interpolation correction) for generating a corrected pixel signal through interpolation using the pixel signals of the adjacent normal pixels of the same color and a method (hereinafter referred to as the gain correction) for multiplying the pixel signal of the phase difference signal by a gain are known (see Japanese Patent Laid-Open Publication Nos. 2012-004729 and 2011-007882, and U. S. Patent Application Publication No. 2011/0109775 (corresponding to Japanese Patent Laid-Open Publication No. 2010-062640)).
The interpolation correction works well with an inconspicuous trace of the correction in the case where a change in a subject is small at around the phase difference pixel. The gain correction works well in the case where a change in a subject is large at around the phase difference pixel. In order to correct the pixel signal of the phase difference pixel, a proper correction method is selected in accordance with the characteristics of the subject located at or around the phase difference pixel.
For example, in the Japanese Patent Laid-Open Publication No. 2012-004729, a change (edge) in a subject at around the phase difference pixel is detected, and the suitable one of the interpolation correction and the gain correction is used in consideration of a gamma correction process performed afterwards. In the Japanese Patent Laid-Open Publication No. 2011-007882, a change (or the direction of a change) in a subject is detected at around the phase difference pixel, and then the suitable one of the interpolation correction and the gain correction is selected. In particular, in the case where the pixel signal of the phase difference signal is corrected through the interpolation correction, it is disclosed that only the normal pixels located in the direction of the change in the subject are used. In the U.S. Patent Application Publication No. 2011/109775, weights of certain proportions are assigned to the pixel signal obtained through the interpolation correction and the pixel signal obtained through the gain correction, respectively, and then the pixel signals are summed. Thereby a corrected pixel signal is obtained. The proportions of the weights are changed based on the change (or uniformity), in the subject, at around the phase difference pixel.
A recent image capture device which executes a normal mode in which a normal image with high resolution is produced and a pixel combining mode that allows imaging with high sensitivity despite the low resolution has been known (see Japanese Patent Laid-Open Publication No. 2011-250325). In the pixel combining mode, the pixel signals of two or more pixels are combined with each other to generate a pixel signal of one combined pixel. Combining the pixel signals is referred to as a pixel addition process or a pixel combining process.
In the pixel combining mode, a composite image is produced by combining the pixel signals under a certain rule. For example, the color image sensor combines the pixel signals of two pixels disposed side by side in a certain direction, out of the pixels of the same color disposed close to each other. In the case where this pixel combining process is applied to the phase difference pixel in a like manner, there are three patterns of combinations, depending on the arrangement of the phase difference pixels: a normal pixel and another normal pixel; a normal pixel and a phase difference pixel; a phase difference pixel and another phase difference pixel. In the two patterns in which the pixel signal of the phase difference pixel is added (excluding the pattern in which the normal pixels are combined with each other) out of the three patterns of the combinations, a new combined pixel that is generated by the pixel combining process is a defective combined pixel, so that the correction of the pixel signal thereof is necessary. In other words, in the color image sensor having the phase difference pixels, the number of the defective combined pixels in the pixel combining mode is greater than the number of the phase difference pixels, being the defective pixels, in the normal image. Therefore the pixel combining mode requires a long time to correct the pixel signals as compared with the normal mode. In the Japanese Patent Laid-Open Publication No. 2011-250325, the arrangement of the phase difference pixels in the color image sensor is improved such that the pixel combining process which involves the phase difference pixel is performed by combining the phase difference pixels of the same color. Thus, there are only two patterns of combinations: a normal pixel and another normal pixel; a phase difference pixel and another phase difference pixel.
In the case where the phase difference pixels are disposed consecutively in the image sensor, the pixel signals from the phase difference pixels may not be corrected accurately in the image data (data composed of the pixel signals of one frame) of the normal image. For example, in the case where the interpolation correction is suitable for correcting the pixel signals of the phase difference pixels, the pixel signals of the normal pixels adjacent to the phase difference pixels are used. In the case where the phase difference pixels are disposed consecutively, the normal pixels necessary for the interpolation correction may be in short supply or the direction in which accurate interpolation is allowed may be limited. As a result, the correction accuracy may be degraded and the trace of the correction may become conspicuous.
In the normal mode, in order to ensure the accuracy of the interpolation correction, it is preferred that, at least, all of the pixels of the same color disposed in the closest proximity to the phase difference pixel are the normal pixels. This means that one or more normal pixels of the same color as that of the phase difference pixel are disposed between the phase difference pixels. In this case, however, the pixel signal of the phase difference pixel is combined with the pixel signal of the normal pixel in the pixel combining mode.
Since a pixel which is based on the pixel signal obtained from the phase difference pixel is a defective pixel, a pixel which is based on the pixel signal obtained by combining the pixel signal of the phase difference pixel with the pixel signal of the normal pixel is a defective combined pixel. In this case, the number of the defective combined pixels is increased, resulting in long correction time. Characteristics of the defective combined pixel differ from those of the defective pixel that is based on the phase difference pixel. The pixel signal of the defective combined pixel cannot be corrected accurately by the same method as that for correcting the phase difference pixel. For example, in a composite image obtained in the pixel combining mode, the pixel signal of the defective combined pixel, which is obtained by combining the pixel signal of the phase difference pixel with the pixel signal of the normal pixel across the edge (the boundary across which the pixel signal changes abruptly) of the subject requires long correction time. Furthermore, even if the interpolation correction and the gain correction for correcting the pixel signal of the phase difference pixel are applied to the defective combined pixel, the result of the correction often turns out to be inaccurate.
To prevent the above-described problems, in the Japanese Patent Laid-Open Publication No. 2011-250325, the arrangement of the phase difference pixels is improved to inhibit combining the pixel signal of the normal pixel with the pixel signal of the phase difference pixel. However, it is impossible to inhibit combining the pixel signal of the normal pixel with the pixel signal of the phase difference pixel. Therefore it is required to accurately correct the pixel signal of the defective combined pixel, which is generated by combining the pixel signal of the normal pixel and the pixel signal of the phase difference pixel.